Papers with enrichment process
Dissecting Recall of Factual Associations in Auto-Regressive Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have focused on identifying where factual knowledge is encoded in the network, but little is known about how it is extracted from the model parameters during inference. |
| Approach: | They examine how factual associations are stored and retrieved internally in LMs . they use attention edges to identify critical points where information propagates to the prediction . |
| Outcome: | The proposed model aggregates information about subject and relation to predict the correct attribute . the model “queries” the enriched subject to extract the attribute based on the proposed model . |